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Jeffrey  Worthington-  BIO 

Director  of  Quality  for  the  USEPA  Office  of  Environmental  Information.  Jeff  served  as  the 
Director  of  Quality  USEPA  ORD  National  Risk  Management  Research  Laboratory  (NRMRL) 
and  as  the  Director  of  Quality  Assurance  for  TechLaw,  Inc.  He  is  an  American  Society  for 
Quality  (ASQ)  Certified  Quality  Manager  and  ASQ  Certified  Quality  Auditor.  Jeff  is  an 
elected  ASQ  Fellow,  a  founding  member  of  the  Government  Division,  Past-Chair  of  the  ASQ 
Energy  &  Environment  Division,  and  a  past  member  of  the  ASQ  Division  Affairs  Council. 

He  is  a  founding  member  and  past  Director  of  the  International  Association  for  Information 
and  Data  Quality  (IAIDQ).  Jeff  served  as  Editorial  Board  member  for  Quality  Assurance. 
Science,  and  the  Law,  the  journal  of  Environmental  Forensics.  Environmental  Laboratory 
magazine,  and  Environmental  Testing  and  Analysis  magazine. 


As  a  Federal  Government  since  1994.  Jeff  co-led  a  team  authoring  the  combined  quality  and 
management  system  for  EPA’s  Environmental  Technology  Verification  (ETV)  program.  He 
co-lea  the  EPA  team  developing  EPA’s  Information  Quality  Guidelines.  Jeff  co-authored  peer 
review  journal  papers  receiving  i)the  USEPA  Science  and  Technological  Achievement  Award 
(STAA),  Level  III  for  equating  EPA  policies  and  procedures  to  U.S.  Supreme  Court  Sound 
Science  Criteria  (2002)  and  2)an  STAA  Honorable  Mention  for  developing  electronic 
recordkeeping  QA  parameters  (2006). 


National  Security  Telecommunications  and  Information  Systems  Security  (NSTISSI)  4011 
Certificate  for  information  systems  security  (INFOSEC)  professionals  and  a  Chief 
Information  Officer  (CIO)  certificate  from  the  National  Defense  University  (NDU) 
Information  Resources  Management  College  (IRMC). 


Jeff  served  as  a  Peace  Corps  Volunteer  in  Kpandu,  Ghana  from  1977-1979. 


Peace  Corps  -  50th  Anniversary 


History 

Peace  Corps  officially  established:  March  1, 1961 

Total  number  of  Volunteers  and  trainees  to  date:  200,000+ 

Total  number  of  countries  served:  139 


Information 

30th  Anniversary  - 

http  ://www.peacecorps.gov/index.cfm?shell=about.  fiftieth 

“Peace  Corps  Volunteer  Ghana  1977” 

http://www.facebook.com/pages/Peace-Corps-Volunteer-Ghana- 

1977/184669711330989#  !/pages/Peace-Corps-Volunteer-Ghana- 

1977/184669711330989  ?sk=photos 


DISCLAIMER 


The  opinions  expressed  in  this 
technical  presentation  are  those 
of  the  authors  and  do  not 
necessarily  reflect  the  views  of 


the  US  EPA. 


OVERVIEW 

First  the  basics! 

1990’s  What  was  the  basis  for  DOE  quality 
requirements? 

1994  Why  did  DOE  and  EPA  “harmonize”  quality 
requirements? 

Where  does  quality  fit  into  NRC’s  regulatory  model? 

What  are  the  nuclear  requirements? 

What  are  the  needs  for  environmental  technology  and 
environmental  measurement  quality? 

Can  the  nuclear  approach  be  used? 

New  questions? 


Internal  Control  -  basics 

Internal  control  is  an  integral  component  of  an 
organization’s  management  that  provides 
reasonable  assurance  that  the  following  objectives 
are  being  achieved: 


effectiveness  and  efficiency  of  operations, (for  example,  a 

quality  control) 

reliability  of  financial  reporting,  and 
compliance  with  applicable  laws  and  regulations. 
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Based  on  scientific  method 


Plan-do-check-act  =  hypothesize, 
experiment,  test,  and  improve 

“First”  used  by  engineers 


"Engineering"  quality  basics 

Usually  focused  on  a  hardware  solution  -  an 
“engineered”  tangible  product 

Planning 

Design 

Construction 

Installation 
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environmental  measurement  quality  basics 

Focus  on  measurements  of  natural  systems  -  chemical,  physical, 
biological,  geographical  (x-y-z),  population 

Measurement  planning 

Sampling 

Analysis 

Data  review 

(decision) 


1990's  What  was  the  basis  for  DOE  quality 

requirements? 

NQA  -  1  used  in  DOE  community 
to  accomplish  nuclear 
requirements 


ASQ  Energy  Division  -  Nuclear 
Quality  Systems  Auditor  Training 
Manual  (oased  on  NQA-i) 
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Why  did  DOE  and  EPA  "harmonize"  quality 

requirements? 

Focus  on  DOE  sites: 

•  Environmental  monitoring 

•  Cleanup  plans 

•  DOE  laboratories  to  conduct  environmental  analysis 

Quality  requirements  currently  available 

•  NQA  -  1  -  focus  on  engineering  needs 

•  EPA  QAMS  -  focus  on  environmental  measurements 

ASQ  Energy  Division  became  ASQ  Energy  &  Environment 
Division 
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E-4  accomplish? 


Aligned  nuclear  (engineering)  and  environmental 
quality  programs 

Recognized  need  for  environmental  scientists  to  apply 
quality  practices  to  environmental  technology 

Provided  more  structured  guide  for  measurement 
planning  in  combined  science  and  engineering 
applications 

Highlighted  general  lack  of  guidance  for  applying 
quality  planning  to  environmental  technology 

Introduced  concept  of  planning  for  (environmental) 
quality  associated  with  information  hardware  and 
software 


Served  as  the  basis  for  a  new  EPA  quality  system  based 
on  E-4 

Provided  the  basis  for  further  development  of  quality 
guidelines  and  procedures 


NRC’s  “How  We  Regulate” 
http://www.nrc.gov/about-nrc/regulatory.html 


Operational 

Experience 

•  Events  Assessment 

•  Generic  Issues 


Regulations  and  Guidance 

•  Rulemaking 

•  Guidance  Development 

•  Generic  Communications 

•  Standards  Development 


Support  for  Decisions 

•  Research  Activities 

•  Risk  Assessment 

•  Performance  Assessment 

•  Advisory  Activities 

•  CRGR  Reviews 

•  Adjudication 


Oversight 

•  Inspection 

•  Assessment  of  Performance 

•  Enforcement 

•  Allegations 

•  Investigations 


Licensing, 
Decommissioning 
and  Certification 

•  Licensing 

•  Decommissioning 

•  Certification 


13 


Where  does  quality  fit  into  NRC’s  Regulatory  Model? 
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The  NRC  Approach  -  Documents 

Code  of  Federal  Regulations  (CFR) 

•  Appendix  B  to  10  CFR  Part  50 

NUREG  -  publications  prepared  by  NRC  staff 

•  NUREG/BR  =  brochures  prepared  by  the  NRC  Staff 

•  NUREG/CP  =  conference  proceedings  prepared  by  NRC  Staff  or  Contractors 

•  NUREG/CR  =  conference  proceedings  prepared  by  NRC  Contractors 

•  NUREG/IA  =  publications  resulting  from  international  agreements 

•  STS  =  Standard  Technical  Specification  -  a  collection  of  NUREG  documents  for  each  major  type 
of  engineering  plant 

•  TSTF  =  Technical  Specification  Task  Force  “travelers,”....  Travelers  are  a  process  used  to  initiate  changes  to  the  STS. 
Additional  details  are  available  in  the  endnote. 

•  CLIIP  =  Consolidated  Line  Item  Improvement  Process  -  this  is  a  method  to  streamline  license  amendment  review 
process  involving  TSTF  changes 

•  Risk  Management  Technical  Specifications  -  risk- informed  improvements  to  technical  specifications  are  made  using 
PRA  =  probabilistic  risk  assessment 

Regulatory  Guides 
Office  of  New  Reactors 

•  ITAAC  =  inspections,  tests,  analyses,  and  acceptance  criteria 
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10  CFR  Domestic  Licensing  of  Production  and  Utilization  Facilities 


Part 

Appendix  A  to  10  CFR  Part  50 

Appendix  B  to  10  CFR  Part  50 

10  CFR  Part  21 

Subject 

General  Design  Criteria 

Main  QA  Regulation 

Reporting  of  Defects  and  Noncompliance  (Last  updated  December  i,  2009) 

10  CFR  50.4(b)(7) 

Written  Communications  -  Quality  Assurance  Related  Submittals 

10  CFR  50.34(b)(6)(h) 

10  CFR  50.34(f)(3)(h)  and  10  CFR 

50.34(f)(3)(iii) 

QA  Program  Description 

Requirements  Related  to  the  Incident  at  Three  Mile  Island  (TMI)  (I.F.i  and  I.F.2) 

10  CFR  50.36(c)(5) 

10  CFR  50.54(a)(1) 

10  CFR  50.54(a)(2) 

10  CFR  50.54(a)(3) 

Administrative  Controls 

QA  Program  Requirement 

Submittal  of  QA  Program  Description 

QA  Program  Description  Change  Control 

10  CFR  50.54(a)(4) 

Reductions  in  Commitments  in  QA  Program 

10  CFR  50.55(e) 

Evaluation  of  Defects  and  Failures  To  Comply  Associated  with  a  Substantial  Safety 
Hazard 

10  CFR  50.55a 

10  CFR  50.71(e) 

10  CFR  52. 1 7(a)(xi) 

Codes  and  Standards 

Maintenance  of  Records 

QA  Program  Description  (Early  Site  Permits) 

10  CFR  52.47(a)(8) 

TMI-Related  Requirements  (Design  Certifications) 

10  CFR  52.47(a)(19) 

QA  Program  Description  (Design  Certifications) 

10  CFR  52.79(a)(17) 

TMI-Related  Requirements  (Combined  Licenses) 

10  CFR  52.79(a)(25) 

QA  Program  Description  (Combined  Licenses) 
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Regulatory  Guides 

Division  1  -  Power  Reactors 
Division  2  -  Research  and  Test  Reactors 

Division  3  -  Fuels  and  Materials  Facilities 

Division  4  -  Environmental  and  Siting 

Division  5  -  Materials  and  Plant  Protection 

Division  6  -  Products 
Division  7  -  Transportation 
Division  8  -  Occupational  Health 

Division  9  -  Antitrust  and  Financial  Review 

Division  1 0  -  General 
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# 


NRC  Environmental  and  Siting  (Division  4):  Regulatory  Guides  4.1  -  4.20 

Title 

4.1  Radiological  Environmental  Monitoring  for  Nuclear  Power 

4.2  Preparation  of  Environmental  Reports  for  Nuclear  Power  Stations 

4.2S1  Supplement  1  to  Regulatory  Guide  4.2,  Preparation  of  Supplemental  Environmental  Reports  for  Applications  To  Renew 
Nuclear  Power  Plant  Operating 

4.3  Measurements  of  Radionuclides  in  the  Environment,  Analysis  of  I-131  in  Milk  (Withdrawn— See  41  FR  53870, 12/09/1976) 

4.4  Reporting  Procedure  for  Mathematical  Models  Selected  To  Predict  Heated  Effluent  Dispersion  in  Natural  Water  Bodies 

4.5  Measurements  of  Radionuclides  in  the  Environment— Sampling  and  Analysis  of  Plutonium  in  Soil  (Withdrawn  —  See  74 
FR  55074,  October  26,  2009.) 

4.6  Measurements  of  Radionuclides  in  the  Environment—  Strontium-89  and  Strontium-90  Analyses  (Withdrawn  —  See  74  FR 
55074,  October  26,  2009.) 

4.7  General  Site  Suitability  Criteria  for  Nuclear  Power 

4.8  Environmental  Technical  Specifications  for  Nuclear  Power  Plants  (for  Comment)  (Withdrawn  See  74  FR  21017,  May  6, 
2009.) 

4.9  Preparation  of  Environmental  Reports  for  Commercial  Uranium  Enrichment  Facilities 

4.10  Irreversible  and  Irretrievable  Commitment  of  Material  Resources  (Withdrawn— See  42  FR  59436, 11/17/1977) 

4.11  Terrestrial  Environmental  Studies  for  Nuclear  Power  Stations 

4.12  (Not  published) 

4.13  Performance,  Testing,  and  Procedural  Specifications  for  Thermoluminescence  Dosimetry:  Environmental  Applications 

4.14  Radiological  Effluent  and  Environmental  Monitoring  at  Uranium  Mills 

4.15  Quality  Assurance  for  Radiological  Monitoring  Programs  (Inception  through  Normal  Operations  to  License  Termination)  - 
-  Effluent  Streams  and  the  Environment 

4.16  Monitoring  and  Reporting  Radioactivity  in  Releases  of  Radioactive  Materials  in  Liquid  and  Gaseous  Effluents  from  Nuclear 
Fuel  Processing  and  Fabrication  Plants  and  Uranium  Hexafluoride  Production 

4U7  Standard  Format  and  Content  of  Site  Characterization  Plans  for  High-Level- Waste  Geologic  Repositories 
4_r8  Standard  Format  and  Content  of  Environmental  Reports  for  Near-Surface  Disposal  of  Radioactive  Waste 

4.19  Guidance  for  Selecting  Sites  for  Near-Surface  Disposal  of  Low-Level  Radioactive  Waste 

4.20  Constraint  on  Releases  of  Airborne  Radioactive  Materials  to  the  Environment  for  Licensees  other  than  Power  Reactors 
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Parties  involved  in  Nuclear  Quality 

NRC  Commissioners 

NRC  Organization  and  Staff  Structure 

Nuclear  Procurement  Issues  Committee  (NUPIC) 

•  All  domestic  U.S.  nuclear  utilities  and  several  international  members. 

•  To  improve  supplier  assurance  processes  through  cooperative  efforts  while 
minimizing  utility  operations  and  maintenance  and  improving  plan 
performance 

Nuclear  Engineering  Institute  (NEI) 

•  Policy  organization  for  the  nuclear  technology  industry 

•  350  members  in  19  countries 

American  Nuclear  Society  (ANS) 

•  11,500  member  professional  society 


Nuclear  Procurement  Issues  Committee  (NUPIC) 

Conducts  vendor  assessments 
Results  are  shared  among  members 
NRC  may  accompany  members  of  audit  teams 
Focus  on  the  NUPIC  Checklist 

Items  that  NUPIC  will  focus  on  this  year: 

•  Dedication  of  commercial-grade  software  used  in  safety- 

related  analysis  per  Appendix  B,  Criteria  III 

•  Internal  Audit  Effectiveness  evaluating  ability  of  vendor 
to  self-identify  problems 

•  Counterfeit  and  fraudulent  parts 

Commercial-grade  = 

Commercial-grade  dedication  =  a  process  by  which  a  commercial-grade  item  (CGI)  is  designated  for 
use  as  a  basic  component. 
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NRC's  July  2003  Comparison  to 
ISO  9001 

Identified  lack  in  ISO  9001  in  the  areas  of  “regulatory 
impact  &  compliance” 

Identified  missing  requirements  in  ISO  9001 

NRC  identified  two  approaches  suitable  for  further 
development 

•  specific  controls  for  ISO  9001  certified  suppliers  during 
procurement  and 

•  using  ISO  9001  certified  suppliers  for  procuring 
commercial-grade  items. 
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Evolution  of  Nuclear  QA 

(Faulkner/Wood  DOE  Environmental  Management  www.em.doe.gov) 

1954  AEC  QC-1  QC  for  AEC  weapons  programs 
1958  DOD  Mil-Q-9858A  Quality  specifications 
1962  NASA  NPC  250-1  Quality  standard 

1964  AEC  QRC-82  QC  for  AEC  naval  reactors 

1965  ASME  Section  III  QAfor  Nuclear  Pressure  Vessels 

1 970  AEC  1 0CFR50  Appendix  B  -  QA  Criteria 

1971  ASME  N45.2  Began  QA  Program  development 
1979  ASME  NQA-1  Consolidated  N45.2  program 
1981  DOE  5700.6  DOE  QA  Policy 

1983  ASME  NQA-2  Consolidated  NQA-2  QA  methods 

1987  ISO  ISO  9000  Quality  Management  System 

1990  ASME  NQA-3  QAdata  standard 

1994  DOE  10CFR830.120  Quality  Assurance  Requirements 

1999  DOE  Order  414. 1A  Replaced  DOE  Order  5700. 6C 
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Exercise  -  building  a 
nuclear-powered  facility 


Decide  who  will  do  what  and  how  long  it  will  take  to  build 
Learn  and  understand  construction  methods  and  nuclear  issues 
Make  detailed  drawing  what  is  to  be  built,  get  all  the  permits 
Buy  the  right  materials,  and  the  nuclear  reactor 

Check  delivered  materials,  store  them  securely,  hire  qualified  workers. 

Build  the  nuclear  plant  according  to  drawings;  check  workmanship 

Keep  eyes  open  to  detect  problems  that  occur  and  fix  them 

Maintain  the  permits  and  save  the  receipts 

Step  back  periodically  and  evaluate  overall  progress 

Ask  the  inspectors  to  look  at  the  job 
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Environmental  Measurement  Discipline  MATRIX 


PROJECT 


Project  planning 


Env.  Technology 
Design 

Sampling 

methodology 

Analytical 

methodology 

Assessment 

methodology 

Information 
capture 


ADMIN. 


Conformity  to 

planning 

requirements 

Sampling  standards 

Analytical  standards 

Assessment 

standards 

Information  and  data 
standards 

Information 

technology 

standards 

Environmental 

technology 

standards 
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Environmental  Measurement  Discipline  MATRIX 


OFFICE 


Agency  mission 
alignment 

Fed  Requirements 

Management 

Integrity 

Fed  Requirements 
IQGs 

Fed  Requirements 
FOIA 

Fed  Requirements 
FISMA 

Fed  Requirements 


PUBLIC 


Solution  benefit 


Transparency 


Utility  of 
information 

Data  provenance 

Usability  of  data 
(secondary) 

Access  to 
information 
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ANALYSIS  -  Observations  of  NRC  Quality 
Approach  as  a  Model  for  Internal  Control  for 
Technical  Operations 

Requirements  narrowly  focused  on  single  industry 

Requirements  focused  on  quality  for  safety 

Quality  for  measurements  covers  all  measurements  - 
both  technical  and  environmental 

Quality  criteria  for  information  focus  now  on 
commercial  grade  software  -  not  necessarily  entire 
model  for  all  information 

Quality  criteria  is  externally  focused  -  link  to  internal 
Agency  quality  may  not  be  clear 
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CLUSION  1  -  NRC  strengths  for  Internal 


Controls  for  Technical  Operations 


Agency  approval  at  all  major  steps  in  the 
project 

Agency  on-site  oversight 

Formal  vendor  overview  program 

Strong  requirements  for  software  related  to 
safety 

Integration  of  quality  and  safety 
considerations 
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CONCLUSION  2  - 

. comparisons  of  Federal  Quality  Systems  not  easily  found 

OTHER  AREAS  TO  EXPLORE: 

What  are  the  basic  categories  of  government  services? 

What  is  the  general  approach  to  ensuring  the  quality 
of  government  services? 

What  are  all  the  sources  of  government  information? 

What  are  the  basic  categories  of  that  information? 

How  can  alignment  of  the  quality  management 
approach  be  strengthened? 
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EPA  Forum  on  Environmental  Measurements  (FEM)  www.epa.gov/fem 

March  28,  2011  EPA  Policy  to  Assure  Competency  of 
Laboratories,  Field  Sampling,  and  Other  Organizations 
Generating  Environmental  Measurement  Data  under  Agency- 
Funded  Acquisitions 

Organizations  performing  environmental  analysis  for  the  Agency  shall  demonstrate 
their  qualifications  in  the  fields  of  analyses  to  be  conducted,  prior  to  performing  such 
analyses. 


Where  accreditation  or  certification  is  available  for  those  fields  of  analysis,  organizations 

may  submit  documentation  of  existing  accreditations  or  certifications. 


Accreditation/certification  granted  by  an  organization  that  accredits  environmental 
data  operations  to  an  international  consensus  standard,  or  a  state  accreditation  or 
certification  program  acceptable  to  EPA,  or  the  contracted  laboratory’s  participation  in 
the  EPA  Contract  Laboratory  Program  for  those  fields  or  analyses,  shall  be  valid  at  the 
time  of  award  and  must  be  sustained  through  the  life  of  the  period  of 
performance. 


If  accreditation  /  certification  is  suspended  or  revoked  at  any  time  during  the  life  of  the 
period  of  performance,  the  EPA  project  officer  must  be  notified  immediately  to 
ensure  any  potential  impact  on  the  scope  of  work  being  performed  is  addressed 
accordingly. 


Discussion 

And 

Questions 

999999 


QUESTIONS? 


CONTACT  INFORMATION 

Jeffrey  Worthington  -  EPA 
202-566-0995 

worthington.jeffrey@epa.gov 


